Text Mining with Support Vector Machines and Non-negative Matrix Factorization Algorithms by Neelima Guduru a Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Science University of Rhode Island

The objective of this thesis is to develop efficient text classification models to classify text documents. In usual text mining algorithms, a document is represented as a vector whose dimension is the number of distinct keywords in it, which can be very large. Consequently, traditional text classification can be computationally expensive. In this work, feature extraction through the non-negative matrix factorization (NMF) algorithm is used to reduce the dimensionality of the documents. This was accomplished in the Oracle data mining software, which has the NMF algorithm, built in it. Following the feature extraction to reduce the dimensionality, a support vector machine (SVM) algorithm was used for classification of the text documents. The performance of models with SVM alone and models with NMF and SVM are compared by applying them to classify the biomedical documents from a subset of the MEDLINE database into user-defined categories. Since models based on SVM alone use documents with full dimensionality, their classification performance is very good; however, they are computationally expensive. On the data set, the dimensionality is 1617 and the SVM models achieve an accuracy of approximately 98%. With the NMF feature extraction, the dimensionality is reduced to a number as small as 4-100, dramatically reducing the complexity of the classification model. At the same time, the model accuracy is as high as 70 – 92%. Thus, it is concluded that, NMF feature extraction results in a large decrease in the computational time, with only a small reduction in the accuracy.